Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic

On-ramp merging is a challenging task for autonomous vehicles (AVs), especially in mixed traffic where AVs coexist with human-driven vehicles (HDVs). In this paper, we formulate the mixed-traffic highway on-ramp merging problem as a multi-agent reinforcement learning (MARL) problem, where the AVs (on both merge lane and through lane) collaboratively learn a policy to adapt to HDVs to maximize the traffic throughput. We develop an efficient and scalable MARL framework that can be used in dynamic traffic where the communication topology could be time-varying. Parameter sharing and local rewards are exploited to foster inter-agent cooperation while achieving great scalability. An action masking scheme is employed to improve learning efficiency by filtering out invalid/unsafe actions at each step. In addition, a novel priority-based safety supervisor is developed to significantly reduce collision rate and greatly expedite the training process. A gym-like simulation environment is developed and open-sourced with three different levels of traffic densities. We exploit curriculum learning to efficiently learn harder tasks from trained models under simpler settings. Comprehensive experimental results show the proposed MARL framework consistently outperforms several state-of-the-art benchmarks.Comment: 15 figure

The Anti-Inflammatory Effects of a Yin Zhi Huang Soup in an Experimental Autoimmune Prostatitis Rat Model

The present study aimed to investigate the therapeutic effects of the Chinese herbal medicine Yin Zhi Huang soup (YZS) in an experimental autoimmune prostatitis (EAP) rat model. In total, 48 rats were randomly divided into the following four groups (n=12/group): saline group, pathological model group, Qianlietai group, and YZS group. We determined the average wet weight of the prostate tissue, the ratio of the wet weight of the prostate tissue to body weight, tumor necrosis factor-alpha (TNF-α) levels in the blood serum, the expression of inducible nitric oxide synthase (iNOS) in the rats’ prostate tissues, and the pathological changes in the prostate tissue using light microscopy. YZS reduced the rats’ prostate wet weight, the ratio of the prostate wet weight to body weight, and TNF-α levels in the blood serum and inhibited the expression of iNOS in the rats’ prostate tissues (P<0.05). Following YZS treatment, the pathological changes in the rats’ prostates were improved compared with those in the model group (P<0.05). Furthermore, YZS treatment reduced inflammatory changes in the prostate tissue. It also significantly suppressed proinflammatory cytokines, such as TNF-α, and chemokines, such as iNOS, in the rat model of EAP

AI of Brain and Cognitive Sciences: From the Perspective of First Principles

Nowadays, we have witnessed the great success of AI in various applications, including image classification, game playing, protein structure analysis, language translation, and content generation. Despite these powerful applications, there are still many tasks in our daily life that are rather simple to humans but pose great challenges to AI. These include image and language understanding, few-shot learning, abstract concepts, and low-energy cost computing. Thus, learning from the brain is still a promising way that can shed light on the development of next-generation AI. The brain is arguably the only known intelligent machine in the universe, which is the product of evolution for animals surviving in the natural environment. At the behavior level, psychology and cognitive sciences have demonstrated that human and animal brains can execute very intelligent high-level cognitive functions. At the structure level, cognitive and computational neurosciences have unveiled that the brain has extremely complicated but elegant network forms to support its functions. Over years, people are gathering knowledge about the structure and functions of the brain, and this process is accelerating recently along with the initiation of giant brain projects worldwide. Here, we argue that the general principles of brain functions are the most valuable things to inspire the development of AI. These general principles are the standard rules of the brain extracting, representing, manipulating, and retrieving information, and here we call them the first principles of the brain. This paper collects six such first principles. They are attractor network, criticality, random network, sparse coding, relational memory, and perceptual learning. On each topic, we review its biological background, fundamental property, potential application to AI, and future development.Comment: 59 pages, 5 figures, review articl

Oscillation Transition Routes of Buoyant-Thermocapillary Convection in Annular Liquid Layers

There are various oscillation transition routes of buoyant-thermocapillary convection in an annular liquid layer. Three types of transition routes including quasi-periodic bifurcation, period-doubling bifurcation and tangent bifurcation have been observed. In our ground experiments, the depth of liquid layer is in a range of 1.6-2.4 mm. The silicone oil with Prandtl number of 28.6 is selected as the liquid medium. The temperature oscillation is detected by a single-point temperature measuring system and the surface oscillation is measured by a laser displacement-sensor with high resolution. The step-heating mode is adopted in the experiments. Transition routes of temperature oscillation and surface oscillation are studied systematically, and the relationship between them is discussed, too

interactive coupling between a tree and raindrops

This paper presents a novel approach for simulating the dynamic coupling between a tree and raindrops based on physical deformation and fluid simulation. By the approach, tree animation in the rain can be simulated in a two-resolution way: branch motion and leaf motion. The branch is represented by the Euler-Bernoulli beam model, and the leaf petiole is represented by the three-prism elastic model. Interaction coupling liquid motion on the hydrophilic surface with a flexible petiole is well implemented by a special design. To simplify the computation process, instead of the computation-intensive three-dimensional Navier-Stokes equations, shallow water equations are used to simulate the water dynamics together with the whole leaf deformation. Simulation has been also made to various phenomena incurred from the interactive coupling. These include, among others, part of impacting raindrops splashing into the air with the remaining flowing along the slant of the leaf and merging into larger ones or hanging on the blade boundary, with the leaf rebounding and vibrating after the drops fall off the leaf. A level-of-detail approach is exploited to accelerate rendering in views of different distances. The experimental results illustrate that the approach can be applied to efficiently generate realistic details of the interactive coupling between a tree and raindrops. Copyright &copy; 2012 John Wiley &amp; Sons, Ltd.This paper presents a novel approach for simulating the dynamic coupling between a tree and raindrops based on physical deformation and fluid simulation. By the approach, tree animation in the rain can be simulated in a two-resolution way: branch motion and leaf motion. The branch is represented by the Euler-Bernoulli beam model, and the leaf petiole is represented by the three-prism elastic model. Interaction coupling liquid motion on the hydrophilic surface with a flexible petiole is well implemented by a special design. To simplify the computation process, instead of the computation-intensive three-dimensional Navier-Stokes equations, shallow water equations are used to simulate the water dynamics together with the whole leaf deformation. Simulation has been also made to various phenomena incurred from the interactive coupling. These include, among others, part of impacting raindrops splashing into the air with the remaining flowing along the slant of the leaf and merging into larger ones or hanging on the blade boundary, with the leaf rebounding and vibrating after the drops fall off the leaf. A level-of-detail approach is exploited to accelerate rendering in views of different distances. The experimental results illustrate that the approach can be applied to efficiently generate realistic details of the interactive coupling between a tree and raindrops. Copyright &copy; 2012 John Wiley &amp; Sons, Ltd

Fabrication of a highly dispersed Pd-core@Pt-shell electrocatalyst for the oxygen reduction reaction

Core-shell nanostructures have been widely investigated to improve the electrocatalytic performance of platinum. However, organic precursors, surfactants or high temperature are usually necessary during the preparation procedure. Unfortunately, these requirements limit the application of these methods on a large scale. Herein, a Pd-core@Pt-shell nanostructure was fabricated through the reduction of K2PtCl4 by dissociated hydrogen at room temperature without the assistance of either a surfactant or a high-boiling point solvent. The shell thickness of this nanostructure was successfully controlled by varying the amount of K2PtCl4; core-shell nanoparticles with a shell thickness of 0.45, 0.75 and 0.90 nm were obtained, as determined by TEM. The remarkable crystallinity and epitaxial growth of the Pd-core@Pt-shell nanostructure were revealed by HRTEM and EDS. According to ICP and XPS, surface segregation of Pt was established. The impressive ORR performance was attributed to the weak adsorption strength of the OHads species, which resulted from the electron transfer impact between the Pd-core@Pt-shell. The facile and clean preparation method can be used to prepare other core-shell nanostructures under a mild atmosphere. (C) 2017, Dalian Institute of Chemical Physics, Chinese Academy of Sciences. Published by Elsevier B.V. All rights reserved